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The following disclosure relates to communications protocols, encryption 
processes and resource management methods deployed to achieve a decentralized 
collective network environment where non-trusting nodes can share resources and 
oonummicate with an expected qual% of service (QOS). 

Tody's communications leqmrements have outpaced the capabilities of current 
systems. There is an urgent need for better security in both voice and data 
conununications and an increased need to protect the transmission stream. These 
heightened reqiiirements coupled with the need for improved availability and 
reliabilify have brought network interoperability and unification to the forefront 

To date^ other communication and network models have utilized patches at (he 
application and operatii^ system (OS) levd to solve any one of these problems. The 
security patches Jilone have spawned an enture industry providing firewall, VPN 
(Virtual Private Network) and IDS (Intrusion Detection System) enhancements to 
protocol limitations. The demand for improved communications and the limitations 
that plague the current models have spawned a need for a new uniting networking 
model with iundamental changes in structure and architfscture. These changes will 
address the interdependency of security, interoperability, mobility and resource 
management (Including priority and QOS) at tiie protocol level. 

Advancements in radio technology, such as phased array antennas and consumer 
and enterprise adoption of wireless 802.11 access points, Jhave made it possible to 
deploy radio mesh networks. Decentralized public me^ networking requires the 
same changes to the existing network models as those addressed, in above. In 
addition, decentralized public mesh networking requires a new mechanism that 
allows non-trusting paities to share and control network resources without the need 
for central administration. 

Further more, decentralized public mesh networking requires dynamic grouping 
for scaling, intelligent optimal route selection for routing within a structure that can 
[^pport the aggregate inheritance of pennissions, and resource allocation to reduce 
the network control overhead to optimize performance (reduce latency, reduce 
processor requirements and increase throughput and guarantee QOS). 

Parti 
Overview 

Chapter 1 



Overview 

The CoCo network is structured as a hieiarchical mesh networi^ with dynamically 
generated routing tables. Eveiy device on the network is capable of being both an 
endpoint and a forwarder of conununicatigns. The term 'node' is used to leSer to an 
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active component pf the network. Node's may differ in capability, but all must be 
able to follow the basic CoCo routing protocols. Node are connected via any number 
of underlying network protocols. AU underlying networks are represented with one 
of two models, the link model or the star model. There is a hierarchical network 
concept which allows groups of nodes to be viewed ^$ a single group, aUowing the 
(tynamic routing mechanisms to $cale to any size. 

End to end connections between non-directly connected nodes are created via a 
circuit ba$ed mechanism similar to asynchronous transfer mode (ATM). One of the 
two npdes wishing to communicate sends a circuit establishment request This 
request is passed though the network based on the routing tables and the requirements 
of the circuit, and builds the circuit as it goes. Once a circuit is established, ail 
packets traveliiig though the cirpuit take the same path. 

This circuit mechanism gives the nod^s some awareness of the higher level end- 
to-end connections. This allows quality of service (QOS) levels to be attached to 
given data flows. Decisions about QOS policy can be nuide at circuit construction 
time. These can then be forced during course of the circuit's existence. In 
addition, routing is static per circuit, thus once a circuit is created, packet routing is 
simple and efficient 

The information needed to set up circuits; including inter-node connectivity and 
link quality of service data is transfeixed thou^ the network via a hiei^chical 
dynamic routing protocol. Thus each node must be aware only of it*s name and it's 
local connectivity. This information is propagated though the network in a way 
whic^ reduces the amount of data which needs to be maintained in each node, while 
providing enough uiformation for reasonably good circuit construction routing 
choices. 

Cryptographic methods are used throughout the protocol to insure the safety, 
authenticity and correctness of the control data moving between nodes. This allows 
the end-to-end user data transport to be secure, the dynamic routing to be tamper 
resistant, tiie QOS to be cryptographically controlled, and generally prevents nearly 
all attadcs on the network, includii^ many denial of service (DOS) attacks. The 
assumptions of trust are very pessimistic, while still allowing useful work to be done 
among many untrusted nodes. 

Another important concept of the CoCo network model is Uiat of the "document". 
A document is a self contained, cryptographically signed, chunk of data understood 
and used by the network to make decisions. Much of the high level protocols, such as 
circuit establishment and dynamic routing involve the exchange of documents. The 
public key distiibution mechanism operates on documents. 

In addition, documents are used to define pplicy. This includes user and groups, 
as well as resource use policies, and delegation of power. Pglicy documents have 
names, and one can refer to a policy by its name. Many nodes can share the same 
policy, ^nd track policy updates. 

The use of documents requires some mechanism to organize and distribute 
documents tiirpughout the network, and indeed there are protocols within the CoCo 
network model to do just this. In addition, som^ dpcuments, such as routing updates 
have additional ways of moving themselves though the network that do not rely on 
the document transfer protocol. 



1.1 Protocol Stack 
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The protocol stack of the CoCo network model does not fit peifectly into tho OSI 
seven layer network model. This is due to the intentional inteidependence between 
the high level end-to-end connection concept and the hop-by-hop routing choices. 
This allows detailed QOS to be enforced. In addition, security is embedded at a 
lower layer, again in an attempt to gain further functionality and integratioa 

The ba$ic protocol stack is shown in Figure 1.1 
Figure 1.1: CoCo stack organization 



giauro 1.3: CoCo stack organizatinn 
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Intisrface Protocol The tmderlying protocol used for inter*node communications is 
referred to as the interface protocol, and is not defined by this specification, although 
it's requirements are. This protocol need not be consistent throughout the network, 
indeed it may dififer on a node-l^-node or link-by-link basis, and there may even be 
• multiple different inter&ce protocols connecting the same nodes. 

CoCo Link Protocol Immediatdy above the interface protocol is the link protocol. 
This protocol is in charge of the process of initiating CoCo communications with a 
neigihbor, determining the ndg^bors name, establishing basic link ciyptographic 
communications, etc. In addition, this protocol is the base for CoCo's QOS 
mechanisms. 

CoCo Reliability Protocol There is a lightweight reliability protocol that allpws for 
reliable sequenced streams over unreliable packet traffic. This protocol is used by 
both the system level and the user level prptocols, and is used in multiple locations 
within the protocol stadc. 

CqCo Routing Protocol The movement of routing data through the network is 
known as the routing protocol. This protocol is in charge of makmg sure the netwotk 
keeps aware of the current interconnectivity status, as well as the current QOS 
situation. It deals with the network as a hierarchic^ system of nodes, networks, 
networks of networks, etc. 

CoCq Circuit Protocol The establishment of end to end coimections, and the long 
haul transfer of data is handled by the CoCq circuit protocol. This protocol is divided 
into the circuit establishment protocol, which sets up circuits, and the circuit data 
protocol, which moves cOta over cucuits. All user data is transported over the circuit 
data protocol, either as mw packets (unreliable circuit), or as reliable streams by use 
Qf the reliability protocol (reliable circuit). 
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CoCo Document Trjinsfer Protocol The CoCO protocol set makes use of user 
agned data components known as "docmnents" as dLribS £rTSsToc£,^t 

The process of findmg a document by name and moving the document 
«!questingnodeisfcaownasthedocumeiittian$f«rpiptZr ^ ^ 



1.2 Node Concepts 



wLt^iSs^c^l^^''"*''''?'',"'"*'*'^- Both endpoints and intennediate 
pomts are nodes Each no4e has a smgle unique network address. This takes the 

^r. ^^^'^ ^ is used to help determine 5,rSS 

stnictore and ti>e propagation of the dynamic routing data. :nie name is 



coming first, and the most general coming last 

For example: machinei . f loor2 . seattleOf f ice . ibm. die highest level nait 
ni^.^ being of a similar level to the autonomous system nmntefof BOT m 
nammg fonnat. referred to in this document as the dot patti naming foriL isus^^ 
name many aspects of the network, mcliiding usen?. ^rnSSlo^iS.? 
pohcies.ormthiscasenode. In each case th^SylS^SSdS?' 

1.2.1 Inter-Node Connectivity 

Each node is able to communicate with one or more oflier nodes diiecdv These 
nodes are refened to as neighbors, or sometimes as direct nod^ Tte mdeS 
protocol used by neighboring nodes to commmuSeTiSS^J^'S S 

gmdehn^ detemme what underlying transport protocols are appropriate """^^ 

• May be connection oriented or connectioujess. 

• be unreliable or reliable 

• Must fit eiflier die link or star model described below 

• Must be able to packetize data in some way 

• Must have a packet MTU (real orvirtua|)of512 bytes or more 

• duplicate or reorder packets 

™™ JS^ ■ ''^ determining its neighbors, initiating a physical 

connection for connection oriented underlying protocok, and exchanlig SS 
I'^^^^P^su^yiogical neighbors may be done throu^ some dS^^proS 

The two basic types of node connectivify 6«*auwa. 

'^Z.^od^tfZTZ^"''^^^^'''^ An edge is a connection 
SaSe J nSe^tJ^^?^ * ^ pomt-to-point connection. Tlus could be for 
wthm radio pioximify, or a node on the other end of a private Tl 
•me feet that a node Ahas edges to two different nodes, b apd c has noSflg oii 
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whether or not B and C have an edge connecting them. The requirements of a edge 
do inchide bidirectionality and some fonn of packet based transport, although in the 
case of a non-packetized physical link, packets could be easily be introduced via a 
framing mechanism of some sort 

Stars A star is used to represent some form of shared transport mechanism that 
connects a number of nodes in such a way that the transitive property of connectivity 
holds among them. An example of a star would be a number of nodes connected to 
an Ethernet switch. If node A can send packets to node B as well as node C via the 
Ethernet switch, than both B and C can send packets back and forth via the same 
Ethernet switch. Thus a star can be thougfht of as the center of a group of 
communicating nodes. Note, radio networks are usually NOT stars. Another w^ to 
look at stars is as the equivalency classed of connectivity. 

1.2.2 Hierarchical Structure 

The naming of node$ introduces a hierarchical structure to the network, in that 
networks of nodes combine to form networks of networks, and networks of networks 
of networks, etc. A group of nodes, which are part of the same network group, or a 
group of networks, which are grouped together, are often referred to as a meta-node. 
This is because whole networks of nodes can be looked at as a single node, and the 
inter-network connectivity as edges between these meta-nodes. 

The highest level meta-nodes form the global network, or root level network. In 
the Internet model, these would be the networks assigned ASN*$. Each of these root 
level networks, or meta-npdes, can contain interna} structure. The depth of the 
structure before one hits the actual physical node does not have to be consistent 
throughout the networks. That is, a meta-node composed of three smaller nodes, and 
a single individual node can be peers in a network. In terms of dot path node names, 
a.b.Q andx.c could both be nodes» and b . q (a meta-node) and x.c (a real node) 
would be peers in network c. 

One requirement for grouping npdes into a meta-node is that all the nodes m the 
group should be fully connected. That is, for any two chosen nodes in the group, 
there must be a path conisisting of only other nodes within the group, which cormQcts 
them. Obviously node$ which are on the same star make a great grouping. In some 
cases, a meta-node may "split" in that the failure of certain links cause; the meta- 
node's components to no longer be fully connected. The CoCo network attempts to 
deal with this situation, but certain fiultire modes are possible, and one should avoid 
grouping nodes in such a way that one or two key links can split the group*. 

1.2.3 Dynamic Routes 

The dyn^imic routing protocol is in charge of propagating node Qonnectivity and QOS 
data through the network. The hierarchic^ nature of the network and the fully 
connectivity assumption are used to limit the flooding of route messages, and 
minimize the size of the route tables generated, 0ius making certain that the dynamic 
routing scales. ^ 
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13 Document Concepts 

Central to the CoCo network model is the document concept. A document in the 
CoCo networic model refers to a self contained, signed, chipik of data understood and 
used by the networic to make decisions. For example, the public k^ distribution 
mechanism works though documents, the dynamic routing involves the exchange of 
route related documents, the QOS mechanism use documents to define rights and 
delegations of rights for network usage. The actual circuit construction requests sent 
by users are documents.-, 



1.3.1 Documents in General 

• 1.3.1.1 Signatures 

Documents are often signed. This signing uses a public key signature mechanism. 
Interestingly, documents are also used to distribute keys. There is a diain of trust tp 
allow this mechanism to work, as well as a well-known root public key. By signing 
documents, one can be sure the signer "supports" the document What "supports" 
means depeids upon the document in question. If thp document is an identity 
document for example, it means the signer vouches that the identify in question is 
legitimate. If the docum^t is a circuit request, it meaijs the signer is the user 
requesting the circuit 



1.3.1.2 Naming and lyping 

All documents have a type associated with them, and many have a name associated 
with them as well. The type is pne of a predefined Ust of types, and each type has a 
different internal format New dppument types may be added in future versions of tfie 
protpcol. Unknown document types should be ignored when present Documents 
also have a name. This i\ame tal^es the form of a "dot path". This is a dot 
separated list of identifiers, for example this . is . a . t^st. Given a type and a dot 
path, there is usually a single unique document, and the document transfer protocol i$ 
designed to find that document However not all documents are long-term 
documents, for example, circuit request or dynamic routing updates. These 
documents have no names and propagate though some other means, and are caUed 
short term documents. 



1.3.1.3 Time Scope 

All documents also have a tim^ scope. This may include a creation time, and 
certamly mcludes an expiration time. This is the time after which the document 
should no longer be considered relevant For example, dynamic route information 
from a mobile radio device should probably have a fakly quick expiration, as ttie Ume 
m which It is useful is Umited. One thing to note is the assumption that there is fairly 
universal time synchronization ampng nodes. However, it is assumed that while data 
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&om a ^cn node can be time oidered with gieat leliabiHty, inter-node times may l^e 
offbyafewsecondstoammute. One hoar time skews ate considered mao^Se 

1.3.2 Document Types 

There are a number of different document types, and document ^e classes Some of 
Jjese. such as itoitity docmneuts. are used in conjmiction withSther S 
flmtmanyotherdcKnmientsiequireidenti^documentstobeuseft^^ Inad^SJ^e^ 
are a nmnber ofdocument types involved with the document transfer pmtoS 
other word^ documents relatmg to document movement and storage A auick 

^'Ztr^J,°T^'"'^ ^ ^''"'^ descriltions of the 

docum^t format for a given document type are given in the chapter relating to the 
use of the^ocument. with a few very basic documents in the diapter 4 ba^ 



1.3.2.1 Identity Documents 

To allow for a very flexible authorization mechanism, there needs to be a way to 
identafy users, nodes, and other logical miits of activity. In addition, it is valuable to 
be able to group tiiese entities to aUow for a single nde to be applied to an entire class 
of entities. To tins end. a hiet^icfaical identity naming mechanism is defined. Note a 
ai^e hmnan user may have more tiian one identity for die multiple roles they haw 

SL^T^^ ^ ''^'^^ ^ ^ employee of their company, and 

another as a member of a social group. In addition, even within a company fliw may 
have multiple roles for the different departinents. etc, of which they lire a nwmber 
Also, non-human components of die network also have identities, for example ever^ 
node has an identity, which is acfaially its network address 

Each idfflitity is named via a dot path. Both die 'ab.c* and b.C can be vaUd 
SS^il^tr^ time W representing a group, and 'ab.c' representing an 
indm^ or a sub-gioup Each identity has a private and public key a^ociatedtith 

w«3r ^^^^'^ ^™ °' P'«ce of equipment, which the 

Identity represents, and tiie public key is distributed. 

The identily document is a document connecting the identity and tt's public key 
TJe id^bty nanae is die docpment name for d» purpose^ of the documoit tiansfer 
h^SK- 'y ^ be signed by the public key of an identity 

hi^er ni the hierarchy. Thus, a group can choose it's users and subgroups, which can 
in tiffa choose sub-groups of tiiere own. In addition tiiere is a flag assoa^ted with the 
Identity document as to whether die identity is a leaf identityf flat is. oto wWcS 
camiot tove sub-identit^es. The root identity document that is tihe pubUc key 
to sign die c of a. b.c is assumed tp exist on all nodes. y nwwsa 

Tlie details of this document are given in ch^ter 4 

1.3.2.2 Storage Location Documents 

H'Li^^^r^S'^^' is used to find docmnents by pamc on the networit 

A II ^ ^ dgcuments. which link names to acbial nodes 

To this end. there are storage location documents. A storage location document gives 
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Uiat node names of nodes which contain infonnalion about a given sub-tree of 
documem names. For example, there may be a storage location fof Si oorTus 
which pomts to a node that is oqe of XEM's document storage servera 

The name of the storage location document is the tree for which it 
authontauve. The node the document points to should either coiLi^J^Suli 
m that tree or contain storage location reconis for more spedfic^STfo^ 

the same name, allowing for multiple docament storage serves. For a document 

The details of this document are given in diaptcr U 

1.3.2.3 Logical Name Documents 

1^ name docmnents canbe used to give other types of documents a more human 
f- 'fS^o^y structured view of the world Logical name 
doaunente can also be nsed to implement redirection strategies. A lo^cal name 
document is a logical dot path (it's key for lookup), and ^ actual dot oath and 
document type It must be sigaed by au identity equS or higher STtt^ oS^S 
path. It redirects to Its actual dot path. 

The details of this document are given in chapter 11 

1.3.2.4 Route Update Documents 

TTiere are a number of routing related documents passed between nodes during the 
pro«sss of moving routing data AB these documents are unnamed and do cannSt be 
lookup up via the document transfer protocoL 

The details of these documents are given in chq>ter 9 

1.3.2.5 Metric Policy Documents 

^Tll^^ docmnents give nodes information to use in deciding the pennissions 
^«^u^T^^ u "^l establishment and data mo^meii decisions. 

Ba^ Aey store the QOS policies. They each have a dot path name, which 
aHo«« polid^ to reference other policy, and to lookup new policy document^ as old 

t^^Tu <to«»mcnt should be signed by an identity equal 

to or higher than their name. ^ 

The details of this document are glv«i in chapter 10 

1.3.2.6 Circuit Request Documents 

These are unnamed documents (in that they do not have a dot path and are not stored 
ma document stprage server). They contain actual requests for ciicuit constniction. 
THeyshould be signed by the idenUty desiring the circuit to be conaructed 
rhe details of this document are given in chapter 10 
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1.3.2.7 Link Setup Documents 

These are unnamed documents (in that they do not have a dot path and are not stored 
in a document storage server). Th^ are used to set up a link and exchange keying 
infonnation used to secure the link. Th^ should be signed by the ftode initiatine the 
link setup. 

The details of this document are given in chapter 8 
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Part2 
Detailed Description 

Chapter 2 



Conventions and basic types 

. In this chapter, certain conventions used to describe the protocols in this document 
are describe. In addition, we will introduce $ome basic fiindamental data types 
which will be used to build more complex ones. 



2.1 Conventions 



2.1.1 Typing 

In prder to describe both in memory state as well as the logical form of various data 
moved across the network, a simple, semi-formal typing system is intrpduced. Given 
parts of data are represented via various types and there are a number of ways of 
combining simple types into more complex types. The details of in-memory 
representation used in actual implementation are not in^ortant» and indeed the types 
used do not need to be the same, as long as the semantics are preserved. 

2.1.2 Encoding 

In addition to the logical form of types, some types have an encoding as well. Tl|i$ 
encoding represents the way the type is turned into a sequence of bytes for movement 
over the netwoik. The encoding should match the description identically, in order for 
multiple implementations of the protocol to interoperate. 

24-3 Protocol Data Units 

A protocol data unit (or PDU) is a named type which is also a basic conversational 
unit of the protocol. At the lowest level, the protocol can be looked at as the 
exchange of PDUs, and the associated change in state of the various nodes involved 
In addition to these physical PDUs there are also cases where a logical PDU 
description is used to represent an applicatioii programmer interface (API) to lower 
level aspects of the implementatioa A PDU like description is also used to represent 
the high level API that the CoCo protocols export. Thus, interfacing with other 
components a£ the system is looked at as similar to sending and receiving data with 
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other nodes. The actual implementatiQii need not u$e this metaphor, but the 
descriptions all do. 



2J2 Basic Types 



2.2.1 TYPE: Integer types U# and S# 



Description 

The integer type are U8,U16,U32,U64,S8,S16,S32,S64 and represent unsigned and 
signed integers of various size ranges, llie U or S represents unsigned or signed 
re^ectively and the number following the letter represents the number of bits used to 
represent the number. Thus, a U8 can store a number from 0 to 255, and a S16 can 
store a number from -32768 tQ 32767. Note: in geaerai, given the number of bits /i, 

unsigned numbers go from 0 to l^-lj and signed numbers go from -2""^ to 2"~^-l. 
Encoding 

The encoding of the integer types as byte sequences is basic two's complement binary 
representatioa For multi-byte integers, network byte order (big-endian) 
rq>reseniation is used 

2.2.2 TYPE: Varlnteger 



Description 

The variable integer type is used to represent unsigned integers. Logically, it is 
capable of storing a number betnreen 0 and 2^-1. The variable nature comes in it's 
encoding, where N is a $elf-sizing variable size» with smaller numbers taking up less 
space. 

Encoding' 

•7 

If the value of the number if between 0 and 127 (2 -1), the encoding of the value is 
one byte, standard binary integer representatioa If the value is betw^ 128 and 

2^'*-l, it is stored in two bytes. This is a standard network order two byte integer, 
with ihe exception tliat the most significant bit pf the first byte is set to 1. If the value 

is greater than 2^^-l the number is stored as a standard network order four byte 
integer, with the exception that the two highest order bits of the first byte ai^ both set 
to 1. 



2.2.3 TYPE: TimeLocption 



Description 

The Ume location type represents a point in time. That is, a particular moment. For 
example, Jan 2, 2002, 7:30 PM, CST could be one way to represent a time location. 
Time locations represent actual absolute time, not wall tim^ and are thus not effected 
by time zones. 

Encoding 

Time locations are encoded as an S64 representing the number of nanoseconds since 
Jan 1, 2001, UTC. 

2.2.4 TYPE: TimeDuration 



Bescription 

The time duration type defines a relative difference between two time locations, or 
in other words, a duration. 

Encoding 

Time duration is represented as an S64 representing the number of nanoseconds that 
make up the duration. 



2.3 Cryptographic Types 



2.3.1 TYPE: Hash 



Description 

The hash type r^resents a cryptographic hash of a byte sequence. When the 
documentation refers to the hash of a logical structure, it means the hash of the byte 
sequence which i$ the encoding pf the logical structure described. When a hash is 
refeired to as being over a nymber of logical components, it is assumed to be the hash 
of the byte sequence formed by concatenating the encoding of each component in the 
order they are mentioned in the description of what to be hashed. 

In addition a hash may be "keyed" by some value. This means that the key is 
encoded and placed both before and afier the encQding of the object being hashed, 
and the hash value is the result of hashing this new sequence. This keying allows for 
symmetric key signatures, iq that one can verify the sender of a piece of data hashed 
m a keyed manor knew the same hash key. 

Encoding 
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WhUe fiihire implementations of the protocol will support multiple neeotiable 
ciyptographic tiash algorithm choices, the cuirent version uses Sns 
cyptogiaphic hash. The details of turning a sequence of STintole^L^ 
rqiresentmg the resulting 128 bit hash as a sequLce of 16 bSs .te^JftS ^ 
document available tromRSA. oyws IS aescnoed m a 

2.3.2 TYFE: Signature 
Description 

,J^^^^*yPV^P^^^'^P^^'^^eo2^ Generally due to 

the constraints on die size of the data the signature is made over, this sigSe ^If a 
cryptographic hadi^ which is a hash of some section of date. sf3 as the 

t^^^t^ ^ ^ ' P^^^" ^ 'denature can be thought of as bing ovS 
aie entire data block wluch is the input to die h^sh. ««"Buv6r 

Ilncoding 

Wule fiiture implementations of the protocol wUl support multiple negotiable 
pubhc key signature algoriflun choices. fl« current versioJWd the RSA 

^^^fl^"^ J^^ °^ *^ ^80^ ^ representation of tS 

resultmg signature are described in a RSA document « uw 

2.3,3 TYPE: PublicKey 
Description 

ri^^r?'*" ^ represents a public key capable of being used for encoding and 
If^^ "^J^"^^^^ P'*'**' ^ encryption and signing mechanist 
bSh k^ do not use the same key. a public key ^ will contain 

Encoding 

iniplemraitationg of the protocol will support multiple, negotiable 
T^A n^if^T ^ ^^^r ^Sori^m choices, the current version iled the 
RSA public key a^gonthms. The key is a 2048 bit RSA key, encoded as described in 
the appropriate pnor art RSA documentatioa « uescnoea m 

23.4 TYPE: SynunetricKey 



Description 

rt^J^J^J^" ^ reprraents a key which can be used to do enoyption and 
depiyptionusing a symmetric cipher. ^ 
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Encoding 

While futme implementations of the protocol will support multiple, negotiat»le 
symmetric kejr ciphers, the current version use$ DES, the key format of which is 
developed and documented by RS A. 

2.3.5 TYPE: PublicEncoded 



Description 

The public encoded type represent a public key encoding of some data. Generally, 
due to the constraints on the size of the data the encoding is made over, this encoding 
is a symmetric key, which is used to encode other data 

Encoding 

While future implementations of the protocol will support multiple, negotiable 
public key encoding algorithm choices, the current version used the RSA public key 
encoding algorithm. The details of thi$ algorithm and the representation of the 
resulting encoding are described in RSA documentation. 

2.3.6 TYPE: PreSymmetric 



Description 

This type represents the initialization vector or other preamble needed to make the 
symmetric cipher secure against certain attacks. 

Encoding 

While future implementations of the protocol will support multiple, negotiable 
synunetric key ciphers the current version uses DES, the IV format of which is 
described in certain RSA documentatioa 



2A Combining Types 

In order to build complej? type from simpler types, there must be a way of combining 
or modifying types. This section describes the mechanisms used to do just that 
There are really three primaiy complex types, Sequence, IS^t, and Structpre types. All 
of these type$ have a well-d^ned method of encoding, v^ch is related to the method 
of encoding its base type or types. 

2.4.1 Sequence 

A sequence is an ordered list of elements all of the same type. It is represented by 
Seguence<Type> where Type is the type of all the sequence elements. Typical 
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c^eration on a sequence might include inserting and removing elements, finding an 
element based on its location^ iterating thought the elements in order, etc. When 
describing an ofiiset within a sequence, a zero based index is used. 

The in memoiy representation of a sequence is implementation dependent and 
may vaiy from one context to another, even for sequences of the same type, based on 
how they are nmipulated. 

The encoding of a sequence as a sequence of bytes is the concatenation of the 
number of elements, represented as a Varlxiteger, with the encoding of each of its 
elements in order. 

2.4.2 Set 

A set in an unordered collection of elements all of the same type. It is represented by 
$et<Type> where Type is the type of all the set elements. Typical operations on a 
set might include finding elements which meet a certain requirement, set imions and 
intersections, modification of certain elements (which is equivalent tQ removing the 
original element and inserting the modified one). Another operation might b^ the 
transversal of the set in a random order, or some specific order based on a comparison 
function on the elements. Note, like mathematical sets, there cannot be two "copies" 
of identical elements within the set Al} tiie uses of sets described within fiiis 
document avoid this case. 

The in memoiy representation of sets is often quite complex. While the set may 
be simply a collection of elements, the need to find elements based on some property 
quickly often necessitates some form of indexing. Indeed, in this document, sets are 
often used as maps, which relate one element with another, by inserting the key-value 
pairs into the set In addition, the need to transverse the set in some particular order 
•often induces some complexity. However as thi$ is an implementation issue, it will 
not be discussed further. 

The encoding of a set as a sequence of bytes is the concatenation of the number 
of elements, represented as a Varlnteger, with the encoding of every element of 
the s^t, in no particular oider. 

2.4.3 Structure 

A structure consists of a sequential, named list of elements, each of its own type. It is 
used to combine various different parts to form a single whole. A typical definition 
looks like: 

Structure 

^fypeOne NameOne 
TypeTwo NameTwo 

In this case, we have two parts, of types TypeOne and TypeTwo respectively. The 
name3 in the case of real structures help explain the meaning of the structure and give 
a way to refer tQ elements. 
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The encoding of a structure is simply the concatenation of die encoding of all the 
parts of the stnicture in the order given. 



Chapter 3 



Document Encoding 

Document encapsulation is the* name for the process of converting the logical form of 
a document into a sequence of bytes for transfer via some network protocol, either the 
reliability protocol, or in the case of the link protocol/ a special fragmentation 
mechanism. This chapter describes the relation between the logical and "physical" 
form (as the octet stream is referred) of the document 



3,1 Component Types 

First we introduce some common, sub-components of documents and describe how to 
turn them into sequences of bytes 



3.1.1 TYPE: DptPath 



Description 

The dot path is a dot separated list of identifiers. It is used in multiple locations 
within the p^tocols. Each identiSer can be any textual sequence of printable glyphs, 
so long as it does liot contaiq a which is the separating character. The sequence 
must begin and end with an identifier, and the dot pan only appear in the interior. 
Thus "a . b . c . " is illegal as there is a dot at the end with no identifier that follows it 

Encoding 

The dot path is encoded as a Varlnteger representing the number of bytes in its 
UTF-S representation, followed by the Unicode UTF-8 of the actual dot path in a 
textual manner. 

3.1.2 TYPE: DocumentType 



Description 

The document type is an enumeration type. Its definition is as follows: 

[Name [Number - ■ ♦ ■ 

I Identity " |7 ^' ^' ' | 
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StorageLocation 


? 


LogicalNamQ 


7 


ControllQarFlood 


7 


CosignRequest 


7 


IJodeFlopa 


7 


Edge Flood 


7 


NextHopFlood 


7 


InterconnectionFlood 


7 


MetricPolicy 


7 


C± rcui tReques t; 


7 


LinkSetup 


7 



Encoding 

The document type is encoded as a Varlnteger representing the enumeration 
number. 

3.1.3 TYPE: SignatureBlock 



Description 

The signature block represents the signature of some thing, usually a cryptographic 
hash, by a particular identity. Its structure is as folloivs: 

Structure 

DotPath Signerldexit 
Signature SignerSig 

where SignerJtdent is the identity that is signing and SignerSig is the 
cryptographic signature of that particular signer. 

Encoding . 
Standard structure encoding 



3.2 Basic Document Structure 



3,2.1 TYPE: Document 



DcsciiptioD 
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TJe document typs r^resents the general foimat of a document, and the associated 
em»din& A portion of the document, refened to as the document bodTif Kme 
dqjendent portion of the document ooay, is the type 

Structure 

DocumentType Tvpe 

°?tPath Name 

TxmeLocation Expiration 

Bytegequence DocumentBody 

„ f „^ DocumentHash 

Set<SignatureBlock> Signatures 

'.where: 

^"^^lier '^*^''^°^*'*®*^®^™<^"^<tftJw enumeration given in 

^^U6bcJ^^^v°^u^/°'^^'^ For example, identity documents 
wtoch fte pubhc key for a given identity are named with the identic^ 

ftey define. This is also the lookup key for the documem transfer pnrtoMl 
For mmamed documents types, this field is omitted fiom the structure 

Expiration is the time after which this document is no longer valid 

DocumentBody is the main matter of the document, and it's meaning 
depends upon th? Type given eaiiier. It is assumed that each document type 
d^bes a body encoding, and that this encoding is self sizing, thus the m^ 
4ocument decodmg logic will be aware when the document body is done 

DocumentHash is the oyptographic hash of the elements Type^Name (if 
present) Expirat ion,and DocumentBody in that order 

^''^^^''1^^^ ■ t^^ °^ signatuiBs to this document by various 'parties 
Each signature is a pubhc key signing of the hash in DocumentHash. 

Encoding 

Chapter 4 

Basic Document Types 

IS^^^r^^^'^J^ ^'^'^ document type which are used by multtole 
subsystems. Cuirentty. there is only one such doalent type: the iSly^^^ 

4.1 Identity Document 
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^ ^?f"I°«"* « "sed to connect an identity name with the identities pubUc 

document must be signed by an identity higher on the signing See to 

™S^h.^* "L^w P«*5 ^g^g id^T'tity must be proper su^ifthe dot 
path of the Identify document bemg signed For example a.b. c could be signed by 
p.cbutnotd-corx.a.b.c.etc. 

4.1.1 TYPE: IdentityDocumentBody 

Description 

«J]lf V^J^^ identity portion is simply as PublicKey, which is the pubUc key 
of the identity named m tiie docummt name. ^ 



Standard encoding for the PublicKey type. 

Chapters 

Link and Circuit Metric Parameters 

The link and routing protocols collect data for the pmpose of determimng QOS 
capabilities along certam paths. Users constiucting dicuits ask for certain QOS 
prpperties. In addition, there may be other fectors, which the user might want to 
^ arcmt routing, such as security clearan<?e level, or other simito absfract 



m«„^ v.^™^^ mechanism to be used for tiie propagation and 

mampulation of such data, tiie concept of a "mebic parameter- is infroduced ThteS 
an aspect of a cjrcmt or a Mnk which might be a reasonable part of the decision 
proce^ of circuit routing AD QOS parameters feD into the idea of metric pJiaS 
as wen as a number of more abstiact thin^ v^>ui«*^s. 

Ttm rachmettic parameter represents a single measurable quantity of some sort 
^l^^^ bandwidtti would be a mefric parameter, as would latency. Each mcttic 
parameter has it s own uwts in which it is measured. For example bps or 
microseconds for bandwidfli or latency respectively. ' 

In addition, tiiwe is support for two basic type of metiic parameter values 
numenc and symbolic. SymboUc par^eters rep^ some lo£laS^^^ 

^ corporati,^-s mtemal 
use) or ISF (for the mlemet). These aspects may also effect routing choice 

• «^ "^^ations *»votye asking for a requked and desired amount! for 
vanous parameter (for example. I require 64 kbps. but would like 100 kbps). there 
must be an understaiding of which direction is "desirable". Jn tiie case of bandWdUi 
high numbers are desirable, in tiie case of latency, low numbers. In the case of 
^bohc parameters, which are gwen using DotPath naming, more specific or less 
specific nay mcrease desuability. « «»b 
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5.0.1 TYPE: MetricParameter 



Description 

The metric parameter is used to specify which parameter is being discussed. 
To this end, each parameter is given a number, thus this type is an eoumeration. 
Additionally, the metric parameters can be divided into four categories, NumericLow, 
NimiericHigh, SymbolicSpec, SyraboIicGen, where tiie first part names the basic 
type, and the second part specifies which direction is desirable. The type of numeric 
^es is a Varlnteger. The parameter type is a U16, with the first two bits deciding 
the categoiy. An initia] list i$: 



Table of Metric Parameters 



t^ame 


Units 


Type 


Mumber 


Bandwidth 
Latency 


bps * lOp 
microseconds 


NtnnericHigh 
MumericLow 


0 

16384 



Encoding 
Standard U16 encoding 



5^1 Uses 

This metric measurement idea is used for a number of uses, these include: 

• Measuring total and current available capacity/metrics on a both a per-circuit 
and cumulative bases 

• Specifying resource allocation policies. 

• Specifying circuit requirements/desires 

The first is used in the rontmg protocol* ttie second in policy management, the third in 
circuit construction. Uteres for all these are described below 



5.1.1 TYPE: CurrentMetricData 

This type ^ve the total and current metric data of an edge. Included are the total and 
current best that a single path could receive, as well as the total and current 
cumulative over all paths. The structure is as follows: 

Structure 



Me tr i cPqiramet:er 
VarlntegejT 
VarXnteges; 
Varlntp^er 



WhichWetric 
PerPathBestCapact:ity 
PerPathBestUnused 
TotalCapacity 



Var Integer 



TotalUnused 



In the case of symbolic types, the structure is actuaUy: 



Structure 



MetricParameter • 
DotPath 



WhichMetric 
Value 



where there is only' one allowable value for the edge. This may be changed in future 
protocol versions. 

Encoding 

Standard structure encoding, which choice of structure determined by the first 
element 



The metric policy documents determine the division of resources into different uses, 
particularly nodes and users and sub-policies. Each policy has a name, which acts as 
the key for the policy document via the document transfer protocol. The policy then 
divides resources into groups and assigns each of the groups to a user, a node or 
meta-node, or a sub-policy. A given node is assigned a single master policy which is 
used to divide resources going in and out of it 



Description 

Sub-holder type is u$ed to define which of the three types of sub-holders is being 
used. It is a simple enumeration as follows; 

Subholder Enum 



5.2 Metric Policy 



5.2.1 TYPE: SubholderType 



Mame 



Number 



Sub-Policy 

User 

Node 



7 
? 
7 



Encoding 
Standard US 
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5.2.2 TYPE: MetricPolicyComponent 




Description 



StnictqiFe 



Me t r 1 cParame t er 



ParameterType 
ReservedAmount 
AbpolutePriority 
RelativeOverflowPriority 



Var Integer 
Vaylnteger 
Varlnteger 



Encoding ^ 
Standard structure encoding. 



5-2.3 TYPE: MetricPoUcyPart 



Description 

The policy part represents one of the eventual sub-holders of the resources granted 
to a policy. It descnbes the absolute and relative privileges of the sub-holder IVs 
structure is as follows: 

Structure 

SubholdeirType Type 

DotPath SubholderName 

Set<MetripPolicyComponent> Params 



Encoding 
Standard structure encoding. 



5.2.4 TYPE: MetricPoUcyPart 



Description 
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This IS the body of the metric policy document The key to a metric poHcy 
docmnent IS Rename of the ppUcy. The poUcy must be signed by an identity more 
general than the policy name. The structure ofa policy is a follows: 

Structure 

Set<MetricPolicyPart Sxibp^rts 



Encoding 

Standard structure encoding. 



5,3 Metric Requirements 

The third and final use of the metric system is specifying requirements during circuit 
constmctioa This 13 done using the following types: 

5.3.1 TYPE: SingleRequirement 

Numeric Case: 

Structure 

MetricParameter Parameter 

Varlnteger Required 

Varlnteger Pesired 

Symbolic Case: 
Structure 

MetricParameter Parameter 



DotPath 



Required 



DotPath Desired 
Encodiog 

Stands^ structure encoding, choice of struchire based on first element 
5.3.2 TYPE: MetricRequirements 
TDesc This type defines the reqwrements ofa circuit establishment 
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Structure 

Set<SingleRequirement> Requirements 



Encoding 
Standard structure encoding. 



Chapter 6 

Interface API Requirements 

6.1 Introduction 

The node caUs a method on the inteifece API, which then uses the underiyine 
nrts^ s native protocol to transmit a packet, or perform some requested operation 
Pa^ and events that enter the inteifece via the native protocols are converted to 
oOIbaeks in the interfece protocol and sent up the CoCo protocol stack. Thus the 
mterfeoe API is the layer of abstraction that aUows many types of physical netmnfcs 
to be dealt with m an equivalent manner. The process of calling API's and leceivine 
callbacks is very similar to sending and receiving packets. 

It is assumed this underlying protocol may have some addressing mechaiiism to 
allow links to multiple nodes to be reached via the same physical/logical interface 
but rt may be a simple point-to-point link as well. These remote addresses ai^ 
mteifece specific and opaque to the higher-level protocols. The underlying protocol 
may be connection orientated, in which case the addresses define the locations to 
connect to, otherwise the addresses define a per-packet destination. The underlvinE 

TJ°^\^^J^ ""St have or be given a well 

d^ned MTU fertile data portion of the packet 

In addition. SOTie inteifeces (such as radio) wiU wish to alert the CoCo network 
l^r of newly discovered remote nodes. Also, spnie interfaces (such as Ethernet) 
wiU support broadcast of some form and should allow some method of node 
resolution, sunilar to ARP. TTie API supports both of these mechanisms. 

6.2 Interface Protocol Base Types 

iSdSd!^'' ^ communication with the inteifece API are 

6.2.1 TYPII: RemoteAcfdress 
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Description 

An opaque representation of the remote address, must be understood by the 
int^ce. This type is never sent over the network, except by the interface itself 
perhaps, and thus does not have an encoding. TTius any structure contaijung it does 
not have an encoding. 



6*2.2 TYPE: SeekRequestID 



Description 

A number used by the higher level protocol to associate requests with responses. 
The in mempiy form is identical to a U32. This typo is never sent over the network, 
except by the interface itself perhaps, and thys does not have an encoding. Thus any 
structure containing it does not have an encoding 



6.3 Read and Write 



6.3.1 API: SendPacket 

Direction: Tolhter&ce 
Description 

This can sends a packet down to the intei&ce to be transmitted, its parameters are: 
Paraqseters 

RemoteAddress Address 
ByteSequence PacketData 

where Address is the remote address to send the packet to, and PaeketData is 
the packet data, which must be smaller than the MTU of the inteifece. 

Errors 

Bad Remote Address Remote address is of the wrong form, or was not 
a remote address which is "current". Current remote addresses include those 
found though seddng or detection, and wluch have not had an address down 
event 



6.3.2 API: ReceivePacket 
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Direction: From Interface 
Description 

This is called by the interface when it receives a packet. It*s parameters are: 
Parameters 

RemoteAddress Address 
ByteSequence PacketData 

where Addi^^ss is the remote address the packet arrived from , and P&cketData is 
the packet d^ which wiU be smaller than the MTU of the inter&ce. 



6.4 Seeking 



6.4.1 API: SeekNode 

Direction: Tolnter&ce 

Description 

ThiscaUsendsapactetdowntotheiiiter&cetobetransiiiitted. It's parameters are: 
Parameters 

SeekRequestID RequestiNumb^r 
DotName NodeName 
TimeDuratlon Timeout 

where RequestNiimber is a U32 used to associate the resulting callbacks with this 
call, NqdeName is the node name of the node being sought, and Timeout is the 
amount of time the node seeking is willing tQ wait to find thp node sought. 

Errors 

Request Ntmiber In Use The request number is already the number of a 
pending seek 

Invalid Node Name Thenodenameisnot structurally valid 

Node Not Seekable The interface knows that the node being sought 

will not be found, for example, the node name is not local to the physical 

netwoik 

6.4.2 API: SeekFouQd 
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Direction: From Inter&ce 



Pescription 

This is called by the inteiifoce wheq it finds a node matching an outstanding request 
After the seek found, the request ID is no longer in use and can be reused. It5 
parameters are: 

Parameters 

SeekRequestID RequestNumber 
Remot^Address Address 

where RequestNumber is the request this response is in relation to, and Address 
is the remote address of the found node. Thi$ remote address is now conent (packets 
can be sent on it). 



6.4.3 API: SeekFailure 

Direction: From Interface 



Bescription 

This is called by the interface when it timej out tiying to seek a node on the bebalf 
of a caller. After the seek £ulure, the request ID is not longer in use and can be 
reused Its parameters aie: 

Parameters 

SeekRequestID RequestNumber 
where Reqtues tNumber is the request this response is in relation to. 



6.5 Detection and Address Down 



6.5*1 API: DetectNode 

Direction: From Interface 
Description 

This is called when the interfece detects another node on the network. This may 
happen when a node enters radio range, or for example when anotfier node on a 
broadcast network sends a seek with the local node name. It's parameters are: 
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Parameters 

RemoteAddress Addres s 

DotPath NodeName 
Bool RemoteActive 

Vfherc Address is the address of the node detected, and NpdeNam^ is the probable 
name of the node. After receiving this call, Ad<Jress is currept (can have packets 
written to it). RemoteActive is a flag which indicates some level of initiative on 
the part of the remote node, and is a hint to not t>egin link negotiation, as the remote 
side intends to. 

6.5.2 API: AddressDown 
Direction: From Interface 
Description 

This is called to alert the node that an address is no longer current and that the node 
should no longer send packets to it It3 parameters are: 

Parameters 

RemoteAddress Address 
where Address is the address whidi is no longer current 



Chapter 7 



Reliability Protocol 



74 Introduction 

Instead of building TCP stream like reliability into a particular layer, a lightweight 
reliability protocol is used throughout the design. The protocol needs room for three 
flag bits, and two 32 bit numbers, a sequence and acknowledge. It turns an 
unreliable, two way, packet transmission mechanism into a reliable one. The 
mechanism is the same as that used by TCP. One difference is since the concept of a 
channel (IP ^ Port) has been separated fi-om the reliability aspect, there is no reason 
to require the two directions of the connection to open at the same time, thus one way 
reliable CQunections are allowed. 

Only one stream in either direction is allowed per lower level two way 
connection. Attempting to start a new connection before receiving a confinnation pf 
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closing of the previous one is not allowed. Tbie virtual FDU definition for the 
reliability layer is given below, the actual components may be placed into the 
underlying packet in any manor desired. 

The details of the reliability protocol are not described in this document due to 
the fact that they are semantically identical to the reliability m^hanisms usQd by 
TCP. 

7.1.1 PDU: ReliableChannel 

ReiiableBoth 



1.2 States 



7.24 TYPE: RelabilitySendState 



Structure 



Chapter 8 



Link Protocol 



8.1 Introduction 

The link protocol is the lowest level of conununications between nodes. It is the first 
line of defense, dealing with the ^ety and management of all inter-^node data, as well 
as being the base constnict from which QOS is derived. The goal of the link protocol 
is are: 

• Verify the identity of the remote node 

• Prevent the reading and/pr modification of data moving over the link 

• Prevent denial of service attacks based on CPU overloading, table overflows, 
and make certain that packet insertion cannot damage the transmission of 
other unmodified packets 

• Define the base QOS measurements used by higher level protocols 

• Create a consistent internnode transmission inechanism. 

• Create a reliable sub-channel for higher level protocols to use -for contipl 
packets 

To this end, the link protocol establishes the identity of both sides and exchanges a 
symmetric k^ used for encryption and per-packet veriiicatioa Because of the 
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computational cpmplexity of the necessary public key mechanisms used during link 
establishment, and the danger of a DOS attack based on CPU starving, a node will 
only spend a portion of it's CPU on link establishment. To prevent legitimate 
attempts from being turned away, a computational challenge is implemented to make 
the DOS attad^er unlikely to be able to send *coirect' requests at a reasonable rate. It 
is assumed that the correctness testing method is &st enough be checked without 
penalty. 

In addition, no state is kept for challenge requests, only for successful challenge 
rcptie3, thus DOS attacks on in memory tables should also be able to be avoided 
Also, cryptographic methods are used to prevent false replies, etc. 

All of this complexity is Used to transfer a single document, known as the Link 
Setup document, which contains the public encrypted and signed symmetric key. 
Qne this document is processed, the link will be considered establishe4 and a reply , 
will be returned. Then the link will be used to transfer symmetrically encrypted, 
ciyptogr^phically checksummed packet^, along with QOS information, which 
conqM)ses the "activated" portion of the link protocol 



8.2 Link Documents 



8,2.1 TYPE: LinkSetupBody 



Description 

The link setup document is used for initial link key exchange. It is an unnamed 
document It should be signed the initiating node. It*5 struauxe a single 
symmetric key, publicly encoded to the destination node. 

Encoding 
Standard PublicEncoded mcoding. 



S3 Link Document Packetization 



8.3.1 TYPE: LinkDocumentPart 



Description 

The liiik setup document, combined with the full identifying documentation of the 
initiating node, all concatenated together irom the link setup documentation. This 
string of bytes is most Jikely too large to fit within the link MTU, and tiius must be 
broken into parts. These parts are then ciyptographically hashed. The initial request 
contains the liash for each of the parts, allowing the acgeptance of the pri^iiajy request 
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to clear the way for the part, without allowing an intervening attacker to forge data 
before tfie key has been exchanged. 
The form of each part is as follows: 



Structure 



U8 

Varlnterger 

VarXnteger 

ByCeSequence 



PartNumber 
P^r toff set 
P^rtSize 
Data 



where: 

PartNmnber 

index. 
PartOffset 



is the integer part number of the current part, 0 based 



is the offset in bytes of the data portion of this part relative 
to the entire link documentation set 
PartSize is the size in bytes of the data portion 

Data is (he actual data of the part, which the cryptographic hash of 

$hould have been sent with the link initial request as described in a l^ter 
section. 



8.4 Link Request Type 



8.4.1 TYPE: LjnkRequest 



Description 

The link request type contains information about the requester and the hashes of the 
documentation to follow. The request will result in challenge as described in the next 
section, and the complexity of the challenge will be up to the receiver. The format of 
the link request type is as follows: 



Structure 

Dot:Path . RemoteNode 
U9 NumberOfParts 
Varlnteger ReguestDocSize 
Sequence<Hash> PartHashes 

where RemoteNode is the node name of the node requesting, NutnberOf Part:s 15 
the number of actual data part? that form the documentation as describes in the 
previous section, and RequestDocSize is the total size of all those parts and 
PartHashes is the list of hashes of each part, in order pf the part number. 
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8.5 Link Challenge Exchange 



The purpose of the link challenge mechanism is to make the leqaester do work before 
the link will ^en dieck the validity of the link documentation, as the storing, 
reassembly* and cxyptographic verification of the link documentation is a non-trivial 
process and allowing any nnvmfied node to force the checking node into doing such 
work could result in resource starvation. 

To this end, a node may request a 'challenge* from the node it wishes to establish 
a link with. The node receiving this request makes a new challenge, which is a CPU 
constrained cryptographic task, and signs the chaUenge. To continue the protocol, the 
requesting node must return the signed diallenge along with the solution. Npte: until 
it receives the solution, the receiving node does not need to store any state, or perform 
9ny large computational functions. Upon receiving an apparent solution, the process 
of checking the solution is reasonably quick. 

The challenge chosen for this version of the protocol is searching for a missing 
portion of data which re$ults in a particular cryptographic hash. So long as the hash 
is strong, this will result in a brute force search of the unknown data portion. Thus 
the receiving node may make the challenge as difficult or simple as is desired by 
chan^g the number of unknown bits. 

To make certain that the challenge was truly chosen by the receiving node, and to 
make sure that the request the challenge is related to is the same as the request the 
challenge was sent in response to, the challenge and the request data aie 
cryptographically hashed with a key known only tp the receiving node. 

8.5.1 TYPE: ]:iinkChallenge 



Description 

The link challenge type is used to r^resent the actual cryptographic challenge. It's 
structure is as follows: 



where Pr^P^rt is the first poitipn of the data used to make the hash Result and 
NumberOf Bits is the number of additional bits in the hashes prc-image. If the 
number of bits is not evenly divisible by eight, it is assumed that the final byte has 
zeros in the low order (less significant) bits. 

Encoding 
Stand^jrd structure encoding 



Structure 



Sequence<U8 > 
U8 

Hash 



PrePart 
NumberOf Qits 
Result 



9.5.2 TYPE: LmkChallengeAnswer 
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Description 

This type represents the answer to a LinkChallenge. It has a type equivalent to 
S$guence<U8>, which is remaining bytes which when combined with the 
PrePart of the liixikchallenge create the correct hash. 

Encoding 
Standard Sequenge<U8> encoding. 



8,6 Link Packet Types 

There are a number of link packet types. Since the link protocol exists immediately 
above the intec&ce protocol, all interfoce packets arriving are assumed to be link 
protocol packets. The first byte determines the type of link packet The names and 
associated numbers of each link packet type are as follows: 



Link Packet l^pe Table 



Name 


Number 


LinklnitialRequest: 




LlnkRequestNAK 


1 


LinkRequestChallenge 


1 


Li nkChal 1 eng^Respons e 


7 


LinklnitiaO^CK 


? 


LinklnitialNAK 


? 


LinkPartSend 


? 


liinkPartNAK 


? 


LiokFinalACR 


? 


LlnkFinalNAK 


? 


Liii3cReliablePacket 


? 


LinkXJnr e 1 i cibl e Packe t 


? 


LinkRe^^t 


7 



8.7 Initial Link Setup Protocol 

The initial link setup protocol uses the challenge response mechanism to prepare the 
receiving node to have the link setup documentation transferred to it, while at the 
same time, forcing the initisiting node to do some computational work. It involves the 
following PDU*s 

8.7»1 PDU: LinkliiitialRcquest 
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Description 
Structure 

XiinkRequest RequestD^ta 



Encoding 
Standard structure encoding 

S.7.2 PDU: LinkRequestNAK 



Description 
Structure 

IjinkReque s t Reque s t:Da t a 



Encoding 
Standard structure encoding 

8.7.3 PDU: LinliRequestCIiaUenge 



Description 
Structure 

LinkReque s t Reque s t Da t a 

LinkChallenge Challenge 

Hash ChallengeSignature 

Hash InitializabionKeyHash 

Hash Finali zat ipnKeyHash 



Encoding 
Standani structure encoding 
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8.7.4 PDU: LjnkChallengeResponse 



Description 



Stnicture 



XiinkRequest 

LinkChallenge 

Hash 



RequestData 

Challenge 

Chal lengeS ignature 

Answer 



liinkCh^l lengeAzisw^iT 



Encoding 
Standard structure eiicoding 

8.7.5 PDU: LinklnitialACK 

Description 
Stnictare 

IiinkRequest RequestData 
Sequence<Byte> InitializationKey 

Encoding 
Standard structure encQding 

8.7.6 PDU: LinklnUialNAK 

Description 
Stnicture 

LinkRequest RequestData 
Sequence<Byte> InitializationKey 

Encoding 
Standard structure encoding 
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8*8 Secondary Link Setup Protocol 

8.8.1 FDU: LinkPartSend 

Description 
Structure 

LizikDocuiaentPart Part 

Encoding 
Standard structure encoding 

.8.8.2 PDU: LinkPartNAK 

Description 
Structure 

U8 PartNuxtiber 



Encoding 
Standard structuic encoding 

8.8.3 PDU: LinkFinalACK 



Description 
Structure 

Seq[uence<Byte> Finalize^tionKey 
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Encoding 

Standard structure encoding 

8.8,4 PDU: LinRFinalNAK 

Description 
Structure 

Sec[uence<Byte> FinalizationKey 

Encoding 
Standard structure encoding 



8.9 Link Use Protocol 



8.9.1 PDU: LinkUnreliablePacket 



Description 



Structure 



PreSynnnetric 



IV 
Seq 

Times tanp 

Hash 

Data 



U16 

Hash 
Data 



Encoding 

Standard structure encoding, but cnciypted with the link negotiated symmetric key 



8.9.2 PDU: LinkReliablePacket 



Description 
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Structure 



eSyirane t ri c 
U16 
U16 
Hash 
U32 
U32 
y32 
Data 



IV 
Seq 

Time stamp 

Hash 

SendSeq 

AckSeq 

Windows! ze 

Data 



Encoding 

Standard stroctuie encoding, but enoypted with the link negotiated symmetric key 



8.9,3 PDU: LinkReset 

Description 
Structure 

PreSynunetric IV 

U16 Seq 

XJ16 Timestamp 

Hash Hash 



Encoding 
Standard structure encoding 



Chapter 9 



Routing Protocol 



9.1 Introduction 

The CoCo routing protocol is designed to give nodes the information necessaiy to 
conjBctly move ciicuit establishmept requests closer to their final destinatioa It 
should also allow nodes to make intelligent choices about the patli used based on Oie 
metric parameters of the various links and the requirements of the circuit request 
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To allow the routing protocol to scale, networks are assumed to be luerardiical in 
organization. Each node has a node name, and the routing first gets the circuit 
request to the most g^eial part of the node name, then as the node gets to the most 
general network, the routing infom^iation is moxe specific, allowing the next portion 
of the node name to be examined This process continues until the node arrives at its 
final destination. 

A network of nodes is collectively called a meta-node. If nodes in two different 
meta-nodes are connected via an edge, then the two meta-nodes are connected via a 
meta-edge. There is a mechanism to combine multiple links between meta-nodes into 
a single meta-edge. Edge and meta-edge information is flooded though the network, 
with certain rules to limit the data to relevant areas. The highest level meta-edge data 
flood everywhere. 

The meta-edge data is generated by special meta-node controller nodes. These 
nodes inust have the meta-node key as well as their node key. There may be more 
than one meta-node controller, as they should all come to the same result, and 
flooding ral6 is controlled via each node,* thus additional meta-node controllers will 
simply send extra updates which will be ignored if they are excessive. 

It would be possible to not have mets-node controllers, however, this would 
require all nodes to contain the meta-node key (bad), or the security of the m^ta-^link 
data to require only the signature of one node in each meta-node of the meta-link, 
which is possibly acceptable, but still more open to compromise than the meta-node 
controller mechanism, since it requires compromise of a controller in each meta-node. 

Additionally, the meta-node controller mechanism allows much of the routing 
computation to be somewhat centralized, while still allowing for full failure handling, 
since there can in theory be any number of meta-node controllers. 

Another point worth noting, all star n^orks must be related to a meta-node. 
This is a limitation of the current protocol description, and may be removed in the 
future, although it may actually be the best design decision, as star networks by nature 
qualify as meta-nodes. In addition, nodes in star networks never exchange network 
routing data for the local network, but only communicate with the meta-node 
controllers to exchange coimectivity information. 



9.2 Routing State 

Before describing the protocol used to moving routing information around, we 
describe state of nodes, as well as meta-node controllers. A node may be a meta-node 
controller for any network level, and possibly more than one at a time. If a node is a 
metaniode controller for a given level it has full routing information for that level. 
Otherwise, a node has partial routing information for a gjven level. All nodes hgve 
full routing information for the local network segment 

The partial routing information is actually derived from tiie full routing 
information. The partial routing information is the only information used to actually 
route circuit requests. We describe the form of the full routing information first, then 
the partial routing information, then the algorithm tp generate the partial infomurtion 
from the fiilL 

9.2.1 FuU Routing State Introduction 
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The fuU routing infonnation consists of all &e edges or meta-edges that are part of 
the network, along with the metric parameters of each edge, and all the nodes or 
meta-node which are part of the network, along with the metric parameters of the 
node-interconnect. For the purpose of this discussion the wo^s node and edge will 
be assumed to possibly mean meta-nodes or meta-edges. 

9.2.2 TYPE: Edgelnformation 



Description 

The edge information type has infonnation about a single edge. The format of the 
information is as follows: 

Structure 

Dot Path NodeA 
Dot Path NodeB 
S^quence<CurrentMetricPata> MetricData 

where NodeA and NodeB are the two nodes that are connected. At least one of these 
nodes must be in the network for which the full routing information is being stored. 
The MetricData contains all the most recent total and current free values for the 
metrics tracked by the edge. 

Encoding 
Standard structure encoding 

9.2.3 TYPE: Nodelnformation 



Description 

The node information type has information about a single node. The format of the 
information is as follows: 

Structure 

Dot Path Node 
Sequence<Currexit;MetridData> Metricpata 

where Node is die node in quesdoit The MetricData describe the internal 
cai^city and current free capacity of the node. Thi§ i$ more relevant for meta-nodes, 
since the ability to move data from one side of the meta-node to the other may bO 
limited Capacity is given in terms of the worst inter-link case. That is, is a meta- 
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node connects to thiee other meta-nodes. A, B, and C, these numbers are the worst 
case of themeta-nodes ability to move datafh)m Ato B, B to C, and Ato C. 

Encoding 
Standard stmaure encoding 

9.2.4 TYPE: FqllRouteTable 



Description 

The full route table contains fiill rputing data for a given network at a given level. 
It's format is: 

Structure 

Set<EdgeInfonnation> Edges 
Set:<NodeIn£onnatiQn> Nodes 

whete Edges is the set of all die edge information for all edges in the netwoxks, and 
Nodes is the set of aU node information for all nodes in the network. 

Encoding 
Standard structure encoding 

9.2.5 Partial Routing State Introduction 

The partial routing state represent the calculated next-hop routing information 
specific to a given viewpoint within the network. Given a final destination (to the 
resolution of the network being routed) it results in a list of the destinations one hop 
from the current location, along with metric information about tiie total path metric 
for each of the next hop possibilities. In addition, in the case of meta-routing data 
there is another table giving all the neighboring meta-nodes, and the edge node$ one 
level lower which have a connection to the neighboring meta-nodes and the metric 
dat^ of that particular lower level edge. 

9.2.6 TYPE: NextHopEntry 



Description 

Hie next hop entry is a particular possible next hop to reach a certain final 
destination and the associated full path calculated metric information. 

Structure 
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i3 



Node • 
Node 

CurrentMetricD^ta 



FinalDestination 

NextHop 

MetricData 



Encoding 
Standard structure encoding 

9.2.7 TYPE: InterconnectionEntry 



Descnptipn 

The interconnection entiy contains the information about a lower level node or 
meta-^ode v/bich acts as an edge node between two meta-nodes. as ivell as the 
associated metric data of the actual e4ge which connectis the meta-aodes It's 
structure is as follows: 



Encodipg 
Standard structure encoding 

9.2.8 TYPE: PartialRouteTable 



Description 

The partial route table contains partial routing data for a given network at a given 
level. It's format is: ^ 

Structure 

Set<NextHopBntry> NextUops 
Set<XnterconnectionBntry> Interconnects 

where WextHops is the set of all the next hop information calculated. It i5 assumed 
that the set is mdexed by the FinalDegtination variable of the 
NextHopEntry structure. It should be reiterated that there may be multiple entries 
that lead to the same final destinaUon. Similarly, the Interconnects variable 
should be mdexed by the NeighborMetaNode variable of the 
InterconnectionEntry type, and again there may be more Uian one entiy. 



Structure 



Node 
Node 

CurrentMetricData 



NeighborMetaNode 

EdgeNode 

MetricData 



43 



Encoding 
Standard structure encoding 

9.2.9 Conversion Algorithm 

Hie conversion algoritfam is in cbaige transforming fall routes into partial routes. To 
do this, a method is needed to com1;»ine parallel and serially arranges QpS metrics. 
Each QOS metric must have a parallel combination algoritlmi (the total QOS between 
two point based the QOS on two parallel routes between the points), as well as a 
serial combination algorithm (the total QOS between two points base on the QOS 
from the initial point to an intermediate point, and the QOS from an intermediate 
point to the final point). For example the parallel algorithm for bandwidth is the sum 
of the two bandwidths, the serial alggrithm is the minimimi of the two bandwidths. 

The conversion ^gorithm determines for each final destination node, and for 
each next hop, all the paths to the final nod^ via the next hop. It uses the QOS 
combination mechanisms to convert the acyclic gr^h of QO& into a sin^e QOS 
value, which is associated with the next hop. 



93 Data Movement Introduction 

There are fopr basic parts to the routing protocol. The first is the meta-node 
controller flooding protocol which lets all the nodes pf a meta-node know about node- 
controllers. The second is the double signing protocol. This is the method by which 
one half of a edge or meta-edge sends a partially signed edge to oOier side for 
checking and co-signing. The third is base data flooding protocol, by which the edge, 
meta-edge, node and meta-node data is flooded to everyone who need to know. The 
fourth is the method by which, after using the base data to compute, meta-node 
controller nodes jflood the calculated partial data to the local network. 

All the routing packets exchanged by nodes are not really packets per say, but 
documents which are sent over the reliable portion of tixe link. As mentioned above, 
most of these document (with the exception of the cosign requests) are transferred by 
flooding. Flooding is the process of broadcasting data to everyone on the networic 
who needs to know. 

The flooding mechanism works as follows: Each node upon receiving a 
document from it's neighbors checks to see if the document is even relevant to it. If it 
is not, it is ignored. Next, the npde checks if it has seen the same document recently. 
This is done though a previous document table. To prevent overflow to the table, if a 
node receives too many dociunents from a neighbor within a given time, it 
occasionally drops packets. Either way. if the node has seen the document 
previously, it drops it If it is a new, relevant document, the npde checks the 
signature. If it is incorrect, it lowers the allowed data rate for the neighbor from 
which it received it, as a ndghbor should never send a bad document, and the 
neighbor may be compromised. If all is well, the node checks which neighbors the 
document 1$ relevant to, and send it to all of those to whom it is likely relevant 
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9.4 Controller Flood Protocol 



9.4.1 TYPE: ControllerFloodBody 



Description 

The controller alert protocol lets nodes know about meta-node controllers. The 
relevancy criteria is whether or nQt the node bdongs to the iqeta-node the controller is 
a meta-node controller for. The format of the document body is 

Structure 

DotName MetaNode 
DotName Controller 

where MetaWode is the netwoik name of the meta-node that the controUer is 
controller for, qnd Controller is the full node name of the controller node. ^ 
Obviously, Controller must be a more specific dot name of MetaNode. The 
document must be signed by the meta-node key for the meta-node in questioa 

JG^n^oding 
Standard structure encoding. 

A given node should remember at least a few controllers for each level of meta- 
node which it belongs to. It should always keep the closest controllers and drop the 
furthet one based oi| hop count 



9.5 Cosigning Protocol 



9.5.1 TYPE: CosignRequestBoay 

This protocol is used to get an edge update signed by the other party. It mvolves the 
transfer of a document known as the CosignRequest docuipent This is described as 

Structure* 

EdgeXnform^tion Edgelnfo 

where BdgeXnf o i$ the edge information from one node being sent to the other to be 
consigned It must be signed by the NpdeA parameter of Edgelnf o. 
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Encoding 
Standard structure encoding. 

The co$ign request documents are directed closer to the network n?mied in the 
NodeB portion of the Edge Info entyy. If the request has already entered the 
network named^ then node uses it's information about the controllers to ^od the 
document closer to the closest Jmown controller. 



9.6 Base Flooding 



9.6.1 TYPE: NodeFloodBody 



Structure 

Nodeln£o:ctnation Nodelnfo 

Must be signed by the node or meta^node described in the Nodeinf o, relevant if the 
name of the node with it's first dement removed is a generalization of the cunent 
node's node name. 

Encoding 
Standard structure encoding. 

9.6.2 TYPE: EdgeFloodBody 



Structnre 

Edgelnformation Edgelnfo * 

Must be signed by both node? or meta-nodes described in the Edgelnfo, relevant if 
either node name with it's fiist element removed is a generalization of the current 
node's node name. 

Encoding 

Standard structure encoding. 



9.7 Calculated Flooding 
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9.7.1 TYPE: NextflopFloodBody 



Structure 

MextHopBntry WextHop 
Encoding 

Standard structure encoding. 



9.7.2 TYPE: InterconnectionFIoodBQdy 



Structure 

InterconnectionBntry Interconnection 

Encoding 
Standard structure encoding. 



Cliapter 10 



Circuit Protocols 

SiSTbS'L'lr'"'?? """^ ^° "^"^"^^ '^"^ ^ to move data though 

10.1 Circuit Routing Tables 
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to the two Qodes, that is, it is used only to sqparate the multiple circuits between two 
nodes. 



10.1.1 TYPE:.CircuiflD 



Description 

An identifier which allows the sqparation of multiple circuits between the same node 

Encoding 
A standard U32 

The route tables store a relation between the two sides of a circuit, and possibly 
internal data for QQS accounting and control. The type r^resenting one side of a 
circuit is: 

10.1.2 TYPE: CircuitSide 



Stnicture 

Circuitip ID 

Interface Remotelnterf ace 

RemoteAddress RemoteNode 

where Remotelnterf ace lepreseats the particular inteij^e this side of the circuit 
goes out of, and RemoteNode is the intei&ce specific RemoteAddress of the 
other node. The ID is the Circuit ID relative to the inter-node connection. 

There is a special interface known as "Local". This is the inteiface .\ised if one 
side of the circuit is local 

Encoding 
Standard structure encoding. 

10.1.3 TYPE: CircuitLink 



Description 

The circuit link defines a relation hetween two circuit ends, along with 
implementatipn specific data to manage the QOS, or other things. 

Structure 

CircuitSjLde SideA 
CircuitSide SideB 
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Opaque 



Data 



where SideA and SideB are the two communicating sides, and Data is the extra 
implementation specific data. 

Encoding 
Standard structure encoding. 

. 10.1.4 TYPE: CircuitRouteTable 



Description 

The circuit route table is a set of circuit links, which is the cmrent s^ of open 
circuits. 

Encoding 

Standard Set<Circuitliiiik> encoding. 



The circuit data protocol is the sole protocol which rides over the unreliable link 
protocol. Ifs fiaming begins immediately in the data section of the unreliable link. 
The circuit data protocol basically consists of the CircuitiD of the circuit, 
followed by the actual data being sent over the circuit The official definition is: 

10.2.1 PDTJ: CircuitData 



wh^re ID is the circuit ED of the incoming $ide of the circuit 

Upon recdving a packd^ the circuit is lookup by the ID and the incommg address 
to find the entiy in the circuit route tabl^. If there is no such cntiy, the packet is 
silently dropped If there is an entiy, the packet is redirected to thp other side of the 
entry, that is, if the packet arrived via SideA it is sent to SideB and if it arrived at 
SideB it is sent tp §ideA, It is sent over the unreliable link protocol if the interface 
in non-local, or if the interface is the specia) local interfiice it is sent to the user. 

If no data travels though the circuit for a given (ime, the node should drop the 
entry in the circuit route' table. Keep-alives may be sent by sending empty (0 data 
length) packets. These are NC3T delivered to the user. 



10.2 Circuit Data Protocol 



Structure 



Circuit ID 
ByteSequence 



ID 

Data 
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10.3 Establishment Protocol 

The establishment protocol is used to setup circuits. It involves the movement of a 
circuit request document thou^t the netivprk over the rejiable link protocol. As the 
circuit request moves, it establishes the circuit. Upon reaching the other side, data 
can be sent both ways. If the receiving side does not have any initial data to send, it 
should send a keep-alive to verify the circuit bidirectionally. 
The circuit request document has the format as follows 

10.3.1 TYPE: CircuitReque$tBody 



Description 

The circuit request body contains the actual circuit request It should be $igned by 
the user requesting the circuit be established. The format is as follows: 

Structure 

DotName 
DotName 

Met:r4cRequireinents 
TJ8 

Publ icEncrypt ed 
where 

DestinationNode is the final destination of the circuit 

Service is the name of the service desired on the other side (similar to a port 

number and implementatiQn specific) 
Requirements are the QOS and other metric requirements of the circuit. 
Flags is the flags for the circuit, currently defined are bit 0 (the lowest bit) 

which is encryption, and bit 1 (second lowest) which is reliability. 
Circui tKey is only part of the structure if the flag for enayption is set 

If so, it is the symmetric encryption key, public key encrypted to the recipient 

Encoding 

Standard structure encoding, except that the la^t variable is optional. 

Immediately following the request document on the reliable link is a structure 
which represents the unsigned partial metric data fiom previous hops, along with the 
CirciiitDfifom the last hop. The format of this trailer is: 

Structure 

CircuitID ID 
MetiricRequirements Requirements 



DestinationNode 
Service 
Recjuirement s 
Flags 

CircuitKey 



SO 



where the ID is the ID for the previous hop, and the requirement are the remaining 
requirements. For example, the max desired bandwidth may be lower if the previous 
hop could not theoretically support the oiigmal» or the required latency may be less 
based on latency already used. 



10.4 Encryption and reliability 

The enciyption and reliability layers add additional headers to the circuit packets. 
Enciyption comes before reliabili^. The headers look as follows. 

EncryptionHeader 

PreSymmetric iv 

Hash Verification 

where IV is the initialization vector, and Verif icatiion is a ha^ of the data that 
follows it, and is part of the m:iypted block. 

ReliabUityHeader 

S^ndSeq 

^32 AckSeq 

WindowSize 
U8 Flags 

where the meaning of the parts is as described in the reliabiVly protocol section. 



Chapter 11 



Document Transfer Protocol 

The document transfer protocol is (he mechanism used to retrieve a document based 
on it's name and type. The mechanism used is similar to DNS. DTP is run over a 
circuit, it's service name is simply, "DTP". It uses the reliability protocol over the 
circuit protocol to provide reliable stream service. It can be run encrypted or not, 
although it accepting the requested connection is the decision of tlie DTP server. 

Over the DTP link, document requests pre sent, and documents are returned. 
There is a redirection process, which causes the user to be pointed to another DTP 
server. Instead, the primaiy DTP server can also recursively perform addiUonal 
lookups itself and yetum the results directly. This decision is left to flie implementer, 
an4 could be based on the user requesting, the originating npde, etc. 

Itie document request looks as follows: 
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OocnmentSequcst 

Options (Reserved, set to 0) 
DocumentType Type 
DotHame Name 

and arrives over the reli*lelfekexactfy as shown above. 

If (he DTP server knows the value of the conatt documeaat, it returns it, whicli 
simply means sending the docurnent over the teliabteliiik. 

Sherwise. there are two special types of documents which may he used to 
redirect The first is the logical name document 

11.0.1 TYPE: liOgicalNameBody 



S iSgical name document redirects the document which is it's J^/ and the type 
describ^ in it's body to another document of the same ^ with a dififaent k^. It 
must be signed by an idemi^ equal to or more gpneial than it's k^. 

Structure 

DocumentType Type 
DotName RedijreetKey 

where Type is the type of the document that should be redirected, and 
RedirectKey is the new key to redirect to. 

Encoding 

Standard structure encoding. j^.™»«t 
Note- Unlike most documents, there can be more than one reduectiondocmnent 

with the same toy. so long as they are redirecting different types of docmnCTts 

■Z X^o» important mechanism for redirection is fte docmnent location 
mechamsm. The document location mechanism allows certam porUons of the oot 
ift ^o be rrfated to certain EWP serversi If a server does not l^ow the 
Sl^t, Scan redirect toward a server that does. At wm;st. the root DW ^ 
know if not the document, who isthetheauthpritaUvc machmes for the inore general 
^the sub-txee. ml each of those nodes should either know the document or 
another more specific source, uittLl the document is resolved. 

11.0.2 TYPE: DocumentLocatiQiiBody 

This is a pointer for a given portion of the dot path tree to a document seivcr. If there 
are multiple records which are more general than the re«)rd ^^S^^JPj^ 
most speSfic one should be used. All DTP servers should have Ae lo^tion of m» or 
^ore root servers. AU servers authoritative for a given sub-tree Should have aU 
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documents therein, or more specific location documents for all the sub-trees of their 
sub-tree that they do not hold the Mi data for. 

The key of the document location document is the sub-tree served. It must be 
si^ed by an identity equal to or more general than it's key. 

The structure of the document is as follows: 

Stnicture 

Dot:Path NodeName 

where NodeNatne is a node which is authoritative for the sub-tree which is the key of 
the document 

Encoding 
Standard structure encoding. 

There can ^so be more than one document location document for the same key, 
sittce there can be multiple redundant, fully authoritative servers. 



Chapter 12 



Optional Extensions 



12.1 Multicast Extension 

This extension is vsed to allpw a single node to broadcast the same stream of data to 
multiple nodes in an efficient maimer. To allow this, the format of a circuit is 
modified slightly for multicast circuits (which are kept in a separate table from 
normal circuits). At cadt node, a circuit instead of connecting a single 
Circui tside to another single CircuitSide, connects one "xpof' side to one or 
more "branch" sides. 

Jn addition, there is a Channel which is unique to the sending node, and is used 
along with Ihe sending node to key multicast flows for combination purposes. 

12.1.1 DataFloMT 

When circuit data is received on the root side, it is duplicated and sent out to each of 
the branch sides. No actual data flows in the reverse direction (from branch to root), 
but there must be empty keep-alives to keep the circuit open. When ^ keep-alives is 
received via a branch, it is iTorwarded toward the root if there has not been a keep 
alive from that branch for one half the circuit timeout value, otherwise it is dropped. 
When ^ node has not received a keep-alive frqm a given branch for the timeout Ume, 
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it drops the branch. It there are no more branches, the «i!iy m the miijticast table is 
removed. 

12.1.2 Circuit Establishment 

Multicast circuits must be routed from the root outward Thus if a node wishes to 
join the multicast it must send a multicast prerequest document (defined below) to the 
multicast sender requesting that it join the multicast Once this is done, or in the case 
of the multicast sender initiating, the multicast sender sends a Multicast Request 
document, which travels outward. It is routed Ifte a drcuit request with two 
^cqnions: 

1. Upon receiving a multicast request from the same initial sender via the same 
side, a br^ch is made toward the destination, not a new drcuit 

2. Hiere may be an additional routing preference toward send data towaid the 
same path as an existing branch. 

12.1.3 TYPE: MulticastPreRequestBody 



Description 

Send via a non-sender to the sender to request a circuit establishment be made in an 
outgoing direction. It is forwarded toward the SendingNode and should be signed 
by the requesting user. 

Structure 

DotName DestinationNode 
Dot:Name SendingNode 
DotM^e Cliaiinel 
MetricRequirements Requirements 

Flags 

PublicEncrypted CircuitKey 
where 

DestinationNode is the final destination of the dicuit 
SendingNode is the node multicasting, the destination for this pre- 
request 

Channel is the name of the channel desired on the other side 
Requirements are the QOS and other metric requirements of the circuit. 
Flags is the flags for the circuit, curreaay defined are bit 0 (the lowe$t bit) 
wliich is enoyptioa 

CircuitKey is only part of the structure if the flag for encryption is set 

ff so, it is the symmetric enciyption key, public key encrypted to the recipient 

Eqcoding 

StaQdaid structme encodmg, e^ccept tliat the last variable is optional. 
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12.1.4 TYPE: MulticastRcqucstBody 



^SenrviaAesendertoestablidiac^ Signed by the sending node. Followed by 
a PreRequest. either the one sent to initial this request, or one made up by the sender 

Structure 

^^Ph Prer eques tHash 

where the PrerequestHash is the hash of the pre-request which will follow. The 
point of this mechanism is to allow the requesters permissions to be used for the QOS 
proviaonma but to make certain the sending node accepted the multicast request 

Encoding 
Standard structure encoding 

12.1.5 Destruction 

On receiving a circuit taminate request from one of the branches, the circuit 
terminate request is forwanied, and the circuit cjestroyed, if there is only one branch. 
If there is more than one branch, Ae branch in question is simply removes. A circuit 
terminate request from the root is forwarded to all branches, then the cncmt is 
removed. 

12.2 Automatic Splitting 

Automatic splitting is a process by which a larger meta-node can spUt into various 
smaUermeta-nodes. It is useful for large networks. To accomplish this mechanisrn a 
new type of meta-node is made, called a 'logical' meta-node. This dififers from me 
standard •administrative' meta-node. which is established by out of band key 
exchange. The logical meta-node gets it's authority from a number of nodes within 
the logical meta-node. This involves a new document, called the petition document, 
whiph establishes the lo^cal meta-node key, and is signed by many members of the 
same admmistrative meta-node; which the logical meta-node must be a sub-part of. 
Additionally, these same nodes (the signers) are given new identities signed by the 
logical meta-node key. 

This aUows the creation of a logical network controller, which is a node which 19 
promoted to the network controller status by having it's key signed by it's peers as 
per above. These logical network controllers can be ineflfcctive, but not destfuctive. 
The reason is that the two purposes of the network controller; aggregate routes, and 
sig^ing inter-meta-node Imks, both pan be verified externally. In the case pf route 
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aggreg3tion, each node can do route aggregation themselves, and may from tijne to 
time do so to check the cunent logical netwoik contrpUet and deny fiulh^ support 
(let their current signature timeout) of the NC in the future. The infonnation about 
inter-meta-node connectivity will not be propagated unless the other side of the meta- 
node link cosigns, which will not happen if the results are incorrect. 

In the case where nodes determiae that a logical NC is behaving badly, they €»n 
create another, and so long as there is at least one functioning NC within the meta- 
node, ail should work correctly, the malfunctioning (or maUcious) NC wiU have no 

effect , .... , . , ^ 

The actual process of a split involve^ turning an adnmustrabve or logical meta- 
node which is composed directly of node (lowest level) into multiple smaller meta- 
nodes whi<?h will now form the main meta-node (leveling), or breaking a current 
logical meta-node into more than one logical meta-node (<5viding). 

In both these cases, an NC decides that a split is desirable. In the first case, it is 
the higher level NC, and in ^e second, the NC which is sub-dividing. This involves a 
split-request message. This message (which is a document ^gned by tjie NC) is sent 
to the closest administrative NC (if the current NC is a logical NC) for a co-signature» . 
then flooded though the meta-node, and contains information regarding the 
reassignment of the nodes currendy in it's meta-node (which node to which sub- 
group), and the administrative name of the suggested NC's for each sub-group, as 
wdl as the network names. The split request specifies a time when the split is to 
occur, hi the case of a node receiving multiple split requests, the one which is to 
occur the SOONEST is selected, exact identical times are disambiguated via the 
dictionary ordering of the sending NC's node name. 

Before the split is set to occur, nodes should sign the new logical NC*s petition 
document, as well as receive their new identi^r keys from the logical NC. These keys 
are the administratively assigned local portion of the node, followed by the new 
networks full name, and are signed by the NC's petition k^. 
The detailed process \s: 

1. The splitting meta-node's network control sends a spht request directly to the 
closest administrative network controller (unless it is the nearest ANC) 

2. The ANC cosigns the request and floods it 

3. The newly selected LNC's as well as non-endor§ed nodes that so desire, s&id 
a *network-contcoller-petition* message, containing the node's public k^ to 
all nodes involved. 

4. Each node selects one or more candidates, and signs tile petition 

5. Upon receiving the signature, the meta-node in question signs with it*s 
petition key the new identity of the pgde and returns it to the node in question 

6. Upon receiving the minhnum required votes, the candidate node sends it's 
signed petition to the nearest ANC 

7. The ANC signs a l*NC k^ for &e newly appointed candidate, and send it 
bade. 

8. the new LNC floods the local netwoik with it's signed LNC key document, it 
is now a logical node controller. 

9. If none of a node's candidates has achieved a high enough vote count by the 
time the end of the election (the beginning of the split) is set to occur, the 
node chooses one one or more of the candidate^ that has a high er^ougH vole 
count, and signs it'5 petition and receives it's identity from it 

10. If no candidate^ achieves the minimum required voted before the time 
specified, the split is canceled. 
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12.2^1 Division Choice Function 

WbUe the network control which initially decides to split can choose any method it 
wished to decide the organization of the qdit (in the sense of which nodes go to 
which side), there are both Tequixenifint$ and optimization criteria. The requirements 
are that each sub-group in fully internally connected, that is, each member of a sub- 
group can reach every other member without leaving the sub-group. Additionally, 
there may be a number of criteria which makes some breakdowns more desirable. 

To find a good breakdown a modified method of simulated annealing is 
suggested. First, a fitness must be chosen whidi giv^s a value to each possible 
breakdown, an exanxple might be: 

wl{Na- Nl?)2 + 

w2 !Avg(Bandwidth(x) : e )nter)yAvg{Bandw]dth(x) : x e Intra) 

where the set Inter is the set of inter-group connections, and Intra is the set of 
iirtra-group connections, BandwidthQO is a function that measures bandwidth, Wj and 
W2 are weights, and iV^ and A/^^ are the number of nodes in each sub-group. The goal 
is to minimize the value of the fitness function. This fimction must be normalized to 
a value between 0 (be5t)9nd 1 (worst). 

The annealing methods works as follows: 

1 . All nodes are initially put in sub-group A, with the exception of one randomly 
chosen node on the edge of the original group is put in sub-group B 

2. The temperature T is initialized to 1 

3. A random edge iiode (one which 15 on the edge of a sub-group) is chosen for 
evaluation 

4. If the edge node when switched fi^om group A to B leaves the resulting 
breakdown in a legal state (each group fully internally coimected, at least one 
node in each group), the fitness of the original is evaluated, called F^^^ as is 
the fitness of the network with the chosen node switched F^^. 

5. The node is switdied with a probability of 0.5+ y 

6. Stqps 3 to 5 are rqjeated with continuously fplliijgteinperatures. 

7. The final temperahire is fixed at zero, and all edge nodes are evaluated until 
the system stabilizes 

12.2.2 Split document details 

12.2.3 TYPE: SplitRequestBody 
Description 
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Ttds document is sent via the NC defiling to split the network under it's control It 
is signed by the NC for it's trip to the ANC if needed, and oosigned Ipy the ANC 
before it is Hooded. 



Stnicture 



DotPath 

Set<DotParh> 

Set<DotPat:h> 

Set<I>otPath> 

TiraeliOca t ion 

Varlnteger 

U8 



SplittingNode 
Sugges t edNCs 
SubGroupA 
SxibGxoupB 
SplitTiitie 

SplitType 



Encoding 
Standard structure encoding 



12.2.4 TYPE: ControUerPetitionBody 



Description 

A document which represents the candidacy of a given Node in relation to a split 
(which must follow this document). Should be signed by the ^didate. Also sent 
(with the required number of votes as signatures) to the ANC once a candidate has 
gotten sufficient votes. Also sent signed by a single node to represent a vote cast by 
that node 



Structure 

DotPath CandidateName 

SymetricHash HaehQf Spl i tRequest 

PxiblicRey TheKey 



Encoding 
Standard structure encoding 



12.2.5 TYPE: LogicalKeyAssignment 



Description 

A document sent by the a candidate, signed by their logical key. to nodes which 
have voted for it 
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LogicalName 
AclministrativeName 



Encoding 
Standard structure encoding 

12.2.6 TYPE: LogicalNetworkControlAuthorization 



Description 

Sent by the ANC once it is given a Controller Petition signed by the proper number 
ofnode; 

Structure 

Dotpath CandidateName 
Publiclfey TheKey 



Encoding 
Standard structure encoding 



Structure 

DotPath 
DotPath 
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Parts 
Claims 

1. Public Trust Network A method of oiganization nwi-tnisling parties wifli no 
central administration so tliat parties can securely share lesouroes and 
commumcate with an esqwcted quality of service. 

2. Hierarchical Networking System of organizLag multiple nodes in a 
taerarchical relationship slmcture to efficiently intCT communicate in a way 
that optimizes throughput by utilizing hierarchical pathing to piovide the 
shortest route between two nodes. Netwoik has the same structure on every 
level of the hierarchy. The intemode hierarchical relationship is used 

3. Resource Inheritance and Delegation A system that controls resource 
allocation through inheritance aggtegation and hierarchical rights enforcement 
while allowingfer dynamic sub-division and delegation of resource control 

4. Document exchange networking A method of network communication that 
controls node interaction through the exchange of signed document? 

5. Self healing QOS A decentralized system for maintaining resource euaiantees 
(qnahty of service, QOS) dynamically through path reroiW and 
recombination * 

6. Multicast syndtronizatlon A broadcast communication system that allows a 
single source broadcast tp be independently rebroadcast in multiple difEerent 
time name groupings. 

7. Network self authentication A highly scaleable authentication method that 
reqmres fte cUent making a request to a host to securety process portions of 
audientication request itself. 

8. Global root key A method of distributing public and private keys in a way 
which gives each sovereign entity control over key redistribution and allows 
inter Key cooperation when required. 

9. Cryptographic dynamic routing A process of utilizing a set of cryptographic 
pmmtoves (algorithms) to secure and control dynamic routing at the motoool 
level. 

10. Hierarchical dynamic routing A process of choosing communication routes 
dynamically based on a hierarchical relationship of the nodes comprising a 
network. ^ " 

11. Non-DOS link^blishment A method of establishing a link between nodes 
that reduces DOS by requiring the node requesting Uie establishment to 
p^orm a computation Oiat is nunc complex tiian the compotaticm needed to 
perform the key exchange. 

12. Anonymous link level A mettiod of establishing a communications link 
between two nodes tiuough a ayptographic amhentication process, such tiiat 
Qie identities of the two nodes are not externally visible over die link 

12. Hierarchical se(f proving identity A decentialized nu^od of identity 

mhentancelhatisself-autiienticating. 
lA. Extensible aggregate QOS metric? A decenhaUzed mefliod of quantifione 

ag^gate resource availability for node patti? in a network ttiat.optintizcl 

routmg and distiibutes communication load. 
15. Dynamically distributed network policy ' documents A process to allow 

permission based network policy control via ciyptographically signed 
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documents which are antomaticaUy distributed to the relevant nodes within 
thenetwoilc. • • wjiuua 

16. QQSdMsion and delegation A method of assigning ownership and priority to 
netwoik resources which allows resources to be sub divided for use and 
delegatioa 

17. Cryptographic hierarchical inheritance of permission (Priority) A method of 
cr>T)tographicaUy verifying aggregated inheritance of permissions and 
enforcing resource aUpcation by aggregated inheritance priority 

18. AfesA network subdivision (optimized filustering through simulated annealing) 
A decentralized autonomous method of subdividing a netwoik by optimiz^ 
clustenng; through sunulated annealing. 

19. Relative QOS over non QOS link A method of enforcing resource allocation 
levels on bnks between network nodes that utilize protocols that do not 
support QOS m their definition Or unplementatioa A network bridge that 
StadS Q^"^^*^ compatibility with enstingnetwoik protocols that provides 

20. ^ranched circuit group broadcast A method of broadcasting a communication 
betweai nodes of a networic that redistribute the communication as needed 
dependent on the hierarchical relationship of the intended recipients 

21. Decentralize Routing anti-Recursion Technique for preventing circiilar routes 
m decentralized dynamic routing networks. 

22. QOS aggr^ate dynamic routing A mefliod of decentralized routing that 
optimally chooses routes based on the cunent utilization of networic resources 
and the amount of resources requested for a connection 

23. Secure dynamic address assignment A ciyptographic method of dynamicallv 
assigmng local networic addresses. ^ 

24. Doa,ment rate limiting anti-DOS A method of protecting the communication 
of documents ftom DOS (denial of servicQ attacks) by limiting the rate at 
wiuch document communication can occur. 

25. l^nmic link QOS flooding updates) A process by which each node of the 
networic is mformed of the current resource availability of aU links that 
comprise the netwoik with as UiUe inter-node communication as possible. 
This is done through full and partial hierarchical flooding Ifichniques 

26. Secure threshold election A method of partidpanl. candidate.' election 

f *° administnitive contnd of elections on networks 

comprised of non-trustuig nodes. 
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